Optimized image processing for wavefront coded imaging systems

ABSTRACT

An image processing method includes the steps of wavefront coding a wavefront that forms an optical image, converting the optical image to a data stream, and processing the data stream with a filter kernel to reverse effects of wavefront coding and generate a final image. By example, the filter set kernel may be a reduced filter set kernel, or a color-specific kernel. Methods and systems are also disclosed for processing color images, such as by separating color and spatial information into separate channels. Methods and systems herein are for example useful in forming electronic devices with reduced opto-mechanical, opto-electronic and processing complexity or cost.

RELATED APPLICATIONS

This application claims priority to U.S. Patent Application No.60/360,147, filed Feb. 27, 2002 and incorporated herein by reference.

BACKGROUND

Wavefront coding is a set of specialized aspheric optics, detection andsignal processing. In the prior art, the signal processing is dictatedby the aspheric optics such that the spatial blur caused by wavefrontcoding is removed. The aspheric optics “code” the wavefront so that thesampled image is relatively insensitive to misfocus-related aberrations.The intermediate image from the coded wavefront (i.e., the image sampledby the detector) is not sharp and clear. Signal processing processesdata from the detector to remove the spatial blur caused by the asphericoptics, to “decode” wavefront coding.

By way of example, FIG. 1 shows one prior art wavefront coded opticalimaging system 100. System 100 includes optics 101 and a wavefront codedaspheric optical element 110 that cooperate to form an intermediateimage at a detector 120. Element 110 operates to code the wavefrontwithin system 100. A data stream 125 from detector 120 is processed byimage processing section 140 to decode the wavefront and produce a finalimage 141. A digital filter (a “filter kernel”) 130 is implemented byimage processing section 140. Filter kernel 130 consists of filter taps,or “weights,” that are real or integer values with a maximum dynamicrange due to the limits of processing hardware and software, for example128-bit extended floating-point arithmetic, real number mathematics, andscaling/truncation of integer values. Scaling may for example includegeneral series of partial products computations and accumulations.

In the prior art, optical element 110 and detector 120 do notnecessarily match the available processing capability of imageprocessing section 140; accordingly, image degradation may be notedwithin final image 141. If optical element 110 and detector 120 arematched to the available processing capability, it may require a verycomplex and costly hardware implementation in terms of detectorcapability, optical design of element 110, and/or computer processingarchitectures associated with image processing section 140.

SUMMARY OF THE INVENTION

As described herein below, wavefront coding imaging systems aredisclosed with jointly optimized aspheric optics and electronics. In oneexample, the optics and electronics are jointly optimized for desiredoptical and/or mechanical characteristics, such as to controlaberrations, minimize physical lens count, loosen mechanical tolerances,etc. In another example, the optics and electronics are jointlyoptimized with targeted emphasis on electronic parameters, so as toreduce the amount of silicon and/or memory required in hardwareprocessing; this serves to optimize image quality in the presence ofnon-ideal detectors and/or to optimize image quality with a fixedhardware processing solution (e.g., a low cost solution).

An emphasis on optimizing electronic processing is for example importantin imaging applications that involve high unit quantities and dedicatedhardware processing. Examples of such applications include miniaturecameras for cell phones, video conferencing, and personal communicationdevices. By jointly optimizing the optics and mechanics with theelectronic parameters, high quality imaging systems are made that areinexpensive in terms of physical components, assembly, and imageprocessing.

In certain aspects therefore, wavefront coded optical imaging systemsare disclosed with optimized image processing. In one aspect, the imageprocessing is optimized for hardware implementations in which digitalfilters have filter tap values that are a specialized, reduced set ofintegers. These integers in turn affect the processing hardware toreduce complexity, size and/or cost. Examples of such reduced set filtertap values include “power of two” values, sums and differences of powerof two values, and filters where differences of adjacent filter valuesare power of two. The restrictions associated with these example reducedsets operate to reduce the number of multiplications (or generalizedpartial products summations) associated with image processing.

The filter tap values may associate with the spatial location of thefilter kernel. The dynamic range of the filter tap values may also be afunction of the spatial position relative to positioning of the opticalelements and wavefront coding element. In one example, values near thegeometric center of the filter kernel have a larger dynamic range andvalues near the edge of the filter kernel have a smaller dynamic range,useful when the wavefront coding response tends to converge toward smallvalues as distance from the kernel center increases. The filter tapvalues may also be a function of multi-color imaging channelcharacteristics since different wavelength bands respond differentlywhen processed by a detector or a human eye.

By way of example, two easy to describe optical forms of wavefront codedoptics and suitable for system optimization herein may include weightedsums of separable powers, p(x,y)=Sum ai[sign(x)|x|^bi+sign(y)|y|^bi],and the cosinusoidal forms p(r,theta)=Sum ai r^bi*cos(ci*theta+phii).

In one aspect, an image processing method is provided, including thesteps of: wavefront coding a wavefront that forms an optical image;converting the optical image to a data stream; and processing the datastream with a reduced set filter kernel to reverse effects of wavefrontcoding and generate a final image.

The step of processing may include the step of utilizing a filter kernelthat is complimentary to an MTF of the optical image.

The steps of wavefront coding, converting and processing may occur suchthat the MTF is spatially correlated to mathematical processing of thedata stream with the reduced set filter kernel.

In one aspect, the method includes the step of formulating the reducedset filter kernel to an MTF and/or PSF of the optical image resultingfrom phase modification of the wavefront through constant profile pathoptics.

The method may include the step of processing the data with a reducedset filter kernel consisting of a plurality or regions wherein at leastone of the regions has zero values.

The step of wavefront coding may include the step of wavefront codingthe wavefront such that a PSF of the optical image spatially correlatesto the regions of the reduced set filter kernel.

In another aspect, the method includes the step of modifying one of thewavefront coding, converting and processing steps and then optimizingand repeating (in a design loop) one other of the wavefront coding,converting and processing steps. For example, the step of processing mayinclude utilizing a reduced set filter kernel with a weighted matrix.

In another aspect, an image processing method is provided, including thesteps of: wavefront coding a wavefront that forms an optical image;converting the optical image to a data stream; and processing the datastream with a color-specific filter kernel to reverse effects ofwavefront coding and generate a final image.

In another aspect, an image processing method is provided, including thesteps of: wavefront coding a wavefront that forms an optical image;converting the optical image to a data stream; colorspace converting thedatastream; separating spatial information and color information of thecolorspace converted datastream into one or more separate channels;deblurring one or both of the spatial information and the colorinformation; recombining the channels to recombine deblurred spatialinformation with deblurred color information; and colorspace convertingthe recombined deblurred spatial and color information to generate anoutput image. The method may include a first step of filtering noisefrom the data stream. In one aspect, the method generates an MTF of theoptical image such that spatial correlation exists with the first stepof filtering noise. A second step of filtering noise may occur byprocessing the recombined deblurred spatial and color information. Thissecond step of filtering noise may include the step of utilizing acomplement of an MTF of the optical image.

In another aspect, an optical imaging system is provided for generatingan optical image. A wavefront coding element codes a wavefront formingthe optical image. A detector converts the optical image to a datastream. An image processor processes the data stream with a reduced setfilter kernel to reverse effects of wavefront coding and generate afinal image. The filter kernel may be spatially complimentary to a PSFof the optical image. A spatial frequency domain version of the filterkernel may be complimentary to a MTF of the optical image.

In another aspect, an electronic device is provided, including a camerahaving (a) a wavefront coding element that phase modifies a wavefrontthat forms an optical image within the camera, (b) a detector forconverting the optical image to a data stream, and (c) an imageprocessor for processing the data stream with a reduced set filterkernel to reverse effects of wavefront coding and generate a finalimage. The electronic device is for example one of a cell phone andteleconferencing apparatus.

In another aspect, an image processing method includes the steps of:wavefront coding a wavefront with constant profile path optics thatforms an optical image; converting the optical image to a data stream;and processing the data stream to reverse effects of wavefront codingand generate a final image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a prior art wavefront coded optical imaging system;

FIG. 2 shows one wavefront coded optical imaging system with optimizedimage processing;

FIG. 3 shows one other wavefront coded optical imaging system withoptimized image processing;

FIG. 4 shows one other wavefront coded optical imaging system withoptimized image processing;

FIG. 5 schematically illustrates one filter kernel and associated imageprocessing section;

FIG. 6A shows a general set filter kernel and FIG. 6B shows one reducedset filter kernel;

FIG. 7A shows one generalized logic ALU for a general set filter kernel;FIG. 7B and FIG. 7C show alternative specialized logic for a reduced setfilter kernel;

FIG. 8 shows one wavefront coded optical imaging system with anoptimized filter kernel;

FIG. 9A shows one wavefront coded optical imaging system with optimizedimage processing and filter kernel using diagonal reconstruction;

FIG. 9B shows one wavefront coded optical imaging system with optimizedimage processing and filter kernel using circular reconstruction;

FIG. 10 shows one wavefront coded optical imaging system withcolor-optimized image processing and filter kernel;

FIG. 11A shows one wavefront coded optical imaging system with optimizedimage processing with single-shift differential taps and a gradientfilter kernel;

FIG. 11B shows one wavefront coded optical imaging system with optimizedimage processing with scaling taps and a scaling filter kernel;

FIG. 11C shows one wavefront coded optical imaging system with optimizedimage processing with scaling-accumulating taps and adistributive-property filter kernel;

FIG. 11D shows one wavefront coded optical imaging system with optimizedimage processing with accumulating-scaling taps and adistributive-property filter kernel;

FIG. 12 illustratively shows a final image processed by the system ofFIG. 1 and a final image processed by the system of FIG. 2;

FIG. 13 illustratively shows a final image processed by the system ofFIG. 1 and a final image processed by the system of FIG. 2 employing afilter as in FIG. 11; and

FIG. 14 shows frequency response curves for the filter kernels used inFIG. 12 and FIG. 13.

FIG. 15 shows one optical imaging system with optimized color imageprocessing;

FIG. 16A shows one optical imaging system with optimized color imageprocessing;

FIG. 16B illustrates one correlation and color component analysisassociated with FIG. 16A;

FIG. 17-FIG. 23 demonstrate image components through various stages ofprocessing within FIG. 16A;

FIG. 24 shows a block diagram of one electronic device employingcharacteristics of certain wavefront coded imaging systems;

FIG. 25 illustrates select optimization trade-offs in a schematicdiagram, between system components that are jointly optimized in adesign phase to construct the optical imaging system;

FIG. 26 illustrates select profiles for constant profile path optics;

FIG. 27 illustrates select profiles for constant profile path optics;

FIG. 28 illustrates select profiles for constant profile path optics;

FIG. 29 illustrates select profiles for constant profile path optics;

FIG. 30 shows a surface profile, resulting MTF, along and cross pathsurface forms for one profile of FIG. 16;

FIG. 31 shows one other surface profile, MTF, and along and cross pathsurface forms;

FIG. 32 shows one other surface profile, MTF, and along and cross pathsurface forms;

FIG. 33 shows one other surface profile, MTF, and along and cross pathsurface forms relating to one profile of FIG. 26;

FIG. 34 shows sampled PSFs from the example of FIG. 32 and withoutwavefront coding;

FIG. 35 shows sampled PSFs from the example of FIG. 32 and withwavefront coding;

FIG. 36 and FIG. 37 show and compare cross sections through sampled PSFsfrom FIG. 34 and FIG. 35;

FIG. 38 shows an example of one 2D digital filter;

FIG. 39 shows a processed PSF through the filter of FIG. 38;

FIG. 40 illustrates an amount of rank power for the in-focus sampled PSFand the digital filter of FIG. 35 and FIG. 38, respectively;

FIG. 41 shows corresponding MTFs before and after filtering and in thespatial frequency domain; and

FIG. 42 shows one design process for optimizing optics, detector,wavefront coding optics, digital filter (filter kernel), and/or imageprocessing hardware.

DETAILED DESCRIPTION OF ILLUSTRATED EMBODIMENTS

FIG. 2 shows one wavefront coded optical imaging system 200 withoptimized image processing. System 200 includes optics 201 and awavefront coded aspheric optical element 210 that cooperate to form anintermediate image at a detector 220. Element 210 operates to code thewavefront within system 200; by way of example, element 210 is a phasemask that modifies phase of the wavefront. A data stream 225 fromdetector 220 is processed by image processing section 240 to decode thewavefront and produce a final image 241. System 200 has a filter kernel230 with a reduced set of filter kernel values that optimally match theparticular image processing implementation of image processing section240. Filter kernel 230 may be constructed and arranged with the reducedset of kernel values such that a final image 241 of system 200 issubstantially equivalent to final image 141 of system 100. As describedin more detail below, certain examples of the reduced set of kernelvalues (i.e., tap or weight values) suitable for use with filter kernel230 include: (i) powers of two, (ii) sums and differences of powers oftwo, (iii) a difference between adjacent taps restricted to a power oftwo, and (iv) taps with a range of values specified by spatial locationor color of the image being processed.

FIG. 3 shows another wavefront coded optical imaging system 300 withoptimized image processing. System 300 includes optics 301 and awavefront coded aspheric optical element 310 that cooperate to form anintermediate image at a detector 320. Element 310 operates to code thewavefront within system 300. A data stream 325 from detector 320 isprocessed by image processing section 340 to decode the wavefront andproduce a final image 341. Image processing section 340 processes datastream 325 using a filter kernel 330. Detector 320, data stream 325,filter kernel 330 and image processing section 340 are constructed arearranged to optimize system 300 in terms of complexity, size, and/orcost. Final image 341 may be substantially equivalent to final image141, FIG. 1. As described in more detail below, detector 320 and datastream 325 may include diagonal readouts for latter diagonalreconstruction, and/or multi-channel readouts for color filter arraydetectors. Diagonal reconstruction is useful when performing processingon color filter arrays since many color interpolation algorithms operateon diagonal components in the image. Color-specific filter kernels andprocessing offer reduced cost and improved performance as opposed toprocessing all colors equally. Filter kernel 330 may be the same asfilter kernel 230 of FIG. 2; though those skilled in the art appreciatethat filter kernel 230 may include effects of color and directional(e.g., diagonal) convolution, such as described below.

FIG. 4 shows another wavefront coded optical imaging system 400 withoptimized image processing. System 400 includes optics 401 and awavefront coded aspheric optical element 410 that cooperate to form anintermediate image at a detector 420. Element 410 operates to code thewavefront within system 400. A data stream 425 from detector 420 isprocessed by image processing section 440 to decode the wavefront andproduce a final image 441. Image processing section 440 processes datastream 425 using a filter kernel 430. Image processing section 440 maybe optimized such that final image 441 is substantially equivalent tofinal image 141, FIG. 1. Optics 401 and image processing section 440 arejointly optimized for alternative reconstruction algorithms, such asdiagonal processing or processing with a reduced set of kernel values,that facilitate higher performance at lower cost, size, etc.

FIG. 5 demonstrates one embodiment of a filter kernel 530 and an imageprocessing section 540; kernel 530 and section 540 may for example beused within certain implementations of systems 200-400 above (e.g., tooperate as filter kernel 430 and image processing section 440 of FIG.4). As shown, a detector 520 captures an intermediate image of theoptical system, similar to detector 220, 320, 420 above; data stream 525is input to image processing section 540, similar to data stream 225,325, 425 above. Filter kernel 530 is a reduced set filter kernel withvalues restricted to absolute values that are a power of two (includingzero), such as {−2^(N), . . . , −4, −2, 1, 0, 1, 2, 4, . . . , 2^(N)}.As described in more detail below, image processing section 540implements the multiplication for filter kernel 530 through onlyshifting, as illustrated; only this operation is used since filterkernel 530 is limited to powers of two. In one embodiment, filter kernel530 may be represented by the exponent or equivalently the shift forthat coefficient.

More particularly, digital filtering is a sum of products for bothlinear and non-linear filtering. In the example of FIG. 5, filter kernel530 is applied to the intermediate blurred image sampled by detector520. Consider filter kernel 530 and the intermediate blurred image asseparate two-dimensional objects, with filter kernel 530 centered on aparticular image pixel output from detector 520. At every kerneloverlapping with an image pixel, a product of the associated filterkernel value and image pixel is made; these products are then summed.For linear filtering, this sum is the final filtered image pixel. Tocomplete the convolution, filter kernel 530 is then similarly centeredover every other pixel in the entire image to produce a like value foreach pixel. For linear filtering with an N×N two-dimensional filterkernel, therefore, each filtered image pixel has N² products and N²−1sums of these products. Wavefront coded optical imaging systems 200,300, 400 may thus be optimized with respective optics and filter kernelsmatched to particular hardware implementations of the image processingsection so as to reduce the number of multipliers, sums and relatedimplementation costs. Note that a kernel value of 0 requires onlystorage, so controlling sparseness within a kernel also reducesimplementation cost.

FIG. 6A and FIG. 6B show, respectively, a schematic diagram of a generalset filter kernel and a schematic diagram of a reduced set filter kernel(based on power of two values). In FIG. 6A, the general set filterkernel is not limited to coefficients that are powers of two; it thusrequires a generalized arithmetic logic unit (ALU 550) to perform theproduct of each filter coefficient (i.e., the kernel value) and theimage pixel, as shown. The generalized ALU 550 performs an extendedseries of intermediate logic (known as partial products 552) andaccumulations of the intermediate results (at accumulators 554) to formthe final product (pixel out). In FIG. 6B, on the other hand, since thereduced set filter kernel is implemented with power of two kernelvalues, the arithmetic logic used to form the product may be a shifter560. The amount of bits shifted for each image pixel is determined bythe exponent of the power of two filter value. If for example thereduced set filter kernel consists of a sum of two power of two kernelvalues, then only two shifters and one adder are used in the arithmeticlogic. Accumulations of all shifters 560 result in the final product(pixel out). The implementation of FIG. 6B is thus simplified ascompared to the generalized ALU 550 of FIG. 6A since the shifting andadding logic is predefined as a shift-add sequence, and not as ageneralized partial products summation.

Those skilled in the art appreciate that the arithmetic logic of FIG. 6Bmay favorably compare to other multiplication or generalized scalingimplementations. One example is integer-based look-up-table (LUT)multiplication. Short bit-depth ALUs often implement scalar operandswithin the LUT to improve speed and simplify instruction sets since aseries of partial product summations may consume more time and registerspace as compared to LUT resources. Like the general partial productsaccumulator multiplier, the LUT multiplier thus allocates resources forgeneralized scaling operations. The arithmetic logic of FIG. 6B, on theother hand, employs a reduced set of scaling operations throughbit-shifting logic using power of two coefficients. The arithmetic logicof FIG. 6B is therefore many times smaller and faster than thegeneralized ALU 550 needed to process generalized filter kernel values.

FIG. 7A, FIG. 7B and FIG. 7C show further detail of the arithmetic logicin FIG. 6A and FIG. 6B, to compare generalized and specializedarithmetic logical functions used at each tap in a linear filter: FIG.7A schematically illustrates generalized ALU 550; FIG. 7B and FIG. 7Cschematically illustrate alternate configurations of specialized ALU560. In this example, consider a generalized filter set kernel and areduced set filter kernel that range over the positive integer valuesbetween 1 and 128. The generalized set filter kernel can have all valuesbetween 1 and 128. The reduced set filter kernel can only have thevalues in this range that are a power of two, i.e.: 1, 2, 4, 8, 16, 32,64, and 128. Now consider the product of an image pixel p with ageneralized set filter kernel with a value of 7. Using generalized ALU550, a total of three shifts are required, (4+2+1)xp, or as in FIG. 7A,Q=3. These three shifts are needed to form partial products with factors4 (shift of two), then 2 (shift of one), and then 1 (no shift).Implementation of a generalized set filter kernel with a value of 127thus requires seven shifts and seven intermediate sums.

In comparison, using the reduced set filter kernel with power of twovalues, ALU 560A has just one shift. Even a more sophisticatedimplementation of generalized ALU 550, such as a vector machine orpipelined device, would likely require more resources than such a singleshift.

Another specialized ALU 560B of FIG. 7C depicts a shift-subtractsequence for a known non-power of two coefficient. Continuing theprevious example, if the coefficient is 127 for a pixel value p, thenthe sum of powers of two implementation of 127xp is:64+32+16+8+4+2+1=127. In ALU 560B, therefore, the logic subtracts andenables the summation of negative powers of two to proceed as 128−1=127,requiring only shift and one subtraction to produce the final result(pixel out).

FIG. 8 describes one wavefront coded optical imaging system 800 in termsof a one reduced set filter kernel. System 800 includes optics 801 and awavefront coded aspheric optical element 810 that cooperate to form anintermediate image at a detector 820. Element 810 operates to code thewavefront within system 800. A data stream 825 from detector 820 isprocessed by image processing section 840 to decode the wavefront andproduce a final image 841. Image processing section 840 processes datasteam 825 using a filter kernel 830. By reducing the set of filterkernels in filter kernel 830, as a function of value and of spatiallocation, the hardware implementation of image processing section 840may be optimized to optics 801, wavefront coding aspheric opticalelement 810 and detector 820. Regions A, B, C, and D of filter kernel830 illustrate the reduced geometric set. For this example, values offilter kernel 830 are restricted to zero value in regions A and D, to amoderate range of values (or dynamic range) in region B, and to a largerange of values in region C. All particular kernel values within theprescribed ranges are efficiently implemented from power of two values,sums and differences of power of two values, etc.

Wavefront coded optical imaging system 800 is thus optimized to thereduced geometric set of filter kernels implemented in hardware (e.g.,via filter kernel 830 and image processing section 840). Reducedgeometric set kernels are particularly important in systems withspecialized hardware processors are used with a variety of differentwavefront coded optics and detectors, or used with optics that changecharacteristics, like wavefront coded zoom optics. By way of example,optics 801 and optical element 810 adjust the depth of field and alsoadjust scaling within the kernel size. The prescribed dynamic range istherefore achieved by positioning the coefficient values of filterkernel 830 in appropriate spatial location. By designing optics 801 andoptical element 810 such that the scaled kernel values lie withinregions of adequate dynamic range, image processing section 840 isoptimized for such optics 801 and element 810 given the constraints ofthe largest kernel. While different filter kernels have different valueswithin the geometric regions A-D, the arrangement of values is optimizedwith respect to the capabilities of image processing section 840. By wayof illustration, the rotation of optical element 810 (e.g., by 45degrees) would dictate an equivalent rotation of the kernel; this wouldnot be permitted since the B regions would then lie in a processingspace with only near-zero coefficients rather than the moderate dynamicrange of the B region coefficients. The rotation would only be permittedif the processing space and kernel space were both rotated.

FIG. 9A shows one wavefront coded optical imaging system 900A that isoptimized for its detector and data stream readout. System 900A includesoptics 901A and a wavefront coded aspheric optical element 910A thatcooperate to form an intermediate image at a detector 920A. Element 910Aoperates to code the wavefront within system 900A. A data stream 925Afrom detector 920A is processed by image processing section 940A todecode the wavefront and produce a final image 941A. Image processingsection 940A processes data stream 925A using filter kernel 930A. Inthis embodiment, detector 920A and data stream 925A are not read out intypical row and column format; rather, the output format is for examplea diagonal readout. With such a diagonal readout format, imageprocessing section 940A does not utilize rectilinear processing (e.g.,rectangularly separable or rank-N processing) since the image format isnot row and column based. If the data format is diagonal, optics 901Aand wavefront coded aspheric optical element 910A are arranged such thatthe resulting image of a point object (or its point spread function)contains substantially all information along diagonals in a mannerequivalent to the data stream diagonals. The corresponding filter kernel930A and image processing section 940A then optimally act on image data925A in a diagonal format, as shown; and either, or both, diagonals canbe processed. In one embodiment, the diagonals are remapped torow-column order and operated on in a rectilinear fashion, and thenremapped again in the diagonally ordered output sequence. In oneimplementation, optics 901A and optical element 910A are optimized withimage processing section 940A to provide information in a geometricalorientation that reduces cost. One example is an optical element 910Athat is rotated in a way that corresponds to the diagonal angle of datastream readout 925A.

FIG. 9B shows one wavefront coded optical imaging system 900B that isoptimized for detector and data stream readout. System 900B includesoptics 901B and a wavefront coded aspheric optical element 910B thatcooperate to form an intermediate image at a detector 920B. Element 910Boperates to code the wavefront within system 900B. A data stream 925Bfrom detector 920B is processed by image processing section 940B todecode the wavefront and produce a final image 941A. Image processingsection 940B processes data stream 925B utilizing a filter kernel 930B.Detector 920B and data stream 925B are not of typical rectilinear designand are not read out in typical row and column format. Rather, theoutput format is, for example, an annular or radial-based readout from alog-polar detector. With this readout format, image processing section940B once again does not utilize rectilinear processing (e.g.,rectangularly separable or rank-N processing) since the image format isnot row and column based. If the data format is circular or annular,optics 901B and wavefront coded aspheric optical element 910B arearranged such that the resulting image of a point object (or its pointspread function) contains substantially all information along annular orradial regions in a manner equivalent to data stream 925B. Thecorresponding filter kernel 930B and image processing section 940B thenoptimally act on image data 925B in a circular format, as shown. In oneembodiment, the concentric rings are remapped to row-column order andoperated on in a rectilinear fashion, and then remapped again in thecircular ordered output sequence. In one embodiment, optics 901B andelement 910B may be optimized with image processing section 940B toprovide information in a geometrical orientation that reduces cost.

FIG. 10 describes one wavefront coded optical imaging system 1000optimized for color imaging. System 1000 includes optics 1001 and awavefront coded aspheric optical element 1010 that cooperate to form anintermediate image at a detector 1020. Element 1010 operates to code thewavefront within system 1000. A data stream 1025 from detector 1020 isprocessed by image processing section 1040 to decode the wavefront andproduce a final image 1041. Image processing section 1040 utilizes acolor-specific kernel filter 1030 in processing data stream 1025. Optics1001, aspheric optical element 1010, detector 1020, data stream 1025,filter kernel 1030, and image processing section 1040 cooperate suchthat each color channel has separate characteristics in producing finaloutput image 1041. As a function of color, for example, wavefront codedoptical imaging system 1000 takes advantage of the fact that a human eye“sees” each color differently. The human eye is most sensitive to greenillumination and is least sensitive to blue illumination. Accordingly,the spatial resolution, spatial bandwidth, and dynamic range associatedwith filter kernel 1030 and image processing section 1040 in the bluechannel can be much less than for the green channel—without degradingthe perceived image of image 1041 when viewed by a human. By reducingthe requirements as a function of color, the opto-mechanical andopto-electrical implementation of system 1000 is further optimized ascompared to treating all color channels equivalently.

FIGS. 11A through 11D show related embodiments of a wavefront codedoptical imaging system 1100 that is optimized with specialized filtering(i.e., within an image processing section 1140). Each system 1100A-1100Dincludes optics 1101 and a wavefront coded aspheric optical element 1110that cooperate to form an intermediate image at a detector 1120. Element1110 operates to code the wavefront within system 1100. A data stream1125 from detector 1120 is processed by image processing section 1140 todecode the wavefront and produce a final image 1141. Image processingsection 1140 processes data stream 1125 with a filter kernel 1130.Optics 1101, aspheric optical element 1110, detector 1120 and datastream 1125 cooperate such that a filter kernel 1130 utilizesspecialized filtering and image processing section 1140 utilizesspecialized taps 1145, providing desired cost savings and/or designcharacteristics.

More particularly, the optical systems of FIG. 11A-11D illustrate fourseparate optimizations that may take into account any given coefficient,or sub-group of coefficients, to best implement a particularconfiguration. Any combination or combinations of the embodiments inFIG. 11A through 11D can be implemented with an efficient imageprocessing section.

FIG. 11A specifically shows one embodiment of a gradient filteringoperation. Optics 1101A, aspheric optical element 1110A, detector 1120Aand data stream 1125A cooperate such that a filter kernel 1130A utilizesgradient filtering and image processing section 1140A utilizesspecialized differential taps 1145A. The reduced set filter kernel offilter kernel 1130A is such that the difference between adjacentcoefficients supports efficient implementation. For example, filterkernel 1130A and image processing section 1140A may be configured suchthat the difference between the first and second tap, between the secondand third tap, between the third and forth tap, and so on, are allpowers of two (or another efficiently implemented value or set ofvalues). If each absolute value difference of the filter kernel 1130A isa reduced set of power of two values, then the geometry of eachimplemented filter tap is a differential tap 1145A. Differential filtertap 1145A is an efficient implementation because only a single summationis required for any coefficient value in kernel 1130A. In prior artlinear filtering, such as employed by FIR tap delay lines, only theoriginal image pixel value is propagated to the next tap while theprevious scaled value is not available to the next stage. Recovery ofthe previously scaled value 1146A in the differential tap 1145A allows asingle addition or subtraction to generate the next value. Since theprior result 1146A is propagated to the next tap, a savings is furtherrealized in the size and cost of the hardware implementation of imageprocessor section 1140A. Other combinations of additions andsubtractions can be combined in a differential tap 1145A; a differentialtap 1145A is not limited to a single addition (or subtraction), ratherthose skilled in the art can recognize that any combination of additionsand subtractions can be used in conjunction with carrying forward theprevious result 1146A—it is the carrying-forward action 1146A thatprovides the ability to create an optimal solution, as opposed to theparticular chosen design of the differential tap.

Numerical values provide an even clearer example for a single-adderdifferential tap; consider the coefficient sequence [3, −5, 11, 7, −1,0]. If the input pixel value is p, then the first tap generates 3p. Thenext tap generates −5p, or alternatively using 3p from the previousstep, generates −8p and adds the two values to obtain 3p−8p=−5p. Thecoefficient 11 is similarly obtained from the previous −5p by adding16p. Note that in this case a single shift (p to 16p) and a singleaddition achieve a scaling factor of 11x, whereas traditionalimplementation of 11x based on partial products requires 8+2+1, or twoshifts and two summations. The example continues with 7p=(11p−4p) and−1p=(7p−8p). In some cases, the difference may be zero, in which casethe gradient kernel falls to the detail of the scaling kernel which isdescribed in FIG. 11B.

FIG. 11B shows an embodiment of a scale-filtering operation. Optics1101B, aspheric optical element 1110B, detector 1120B and data stream1125B cooperate such that a filter kernel 1130B utilizes scaledfiltering and image processing section 1140B utilizes specializedscaling taps 1145B. The reduced set filter kernel of filter kernel 1130Bis such that the scale factor between adjacent coefficients supportsefficient implementation. For example, filter kernel 1130B and imageprocessing section 1140B may be configured such that the scale factorbetween the first and second tap, between the second and third tap,between the third and forth tap, and so on, are all factors of powers oftwo (or another efficiently implemented value or set of values). If eachscale factor of the adjacent filter kernel coefficient is a reduced setof power of two values, for example, then the geometry of eachimplemented filter tap is a scaling tap 1145B and can be implementedwith a simple shift on a binary computer (other scale factors may beefficiently implemented by other devices in a more efficient manner thanpowers of 2). Regardless of the scale factor chosen, scaling filter tap1145B is an efficient implementation because only the scaled value of animage pixel at any one tap is passed to both the next tap and the kernelaccumulator. In prior art linear filtering, such as employed by FIR tapdelay lines, the original image pixel value has to propagate to the nexttap while the scaled value is also propagated to the accumulator.Storage of two values (the delayed pixel value plus the scaled pixelvalue) requires two registers. In FIG. 11B, only one register is neededto store the scaled pixel value (scaled from the previous tap) becauseof scaled filtering of filter kernel 1130B. Since the original imagepixel information is not propagated to all subsequent taps, a savings isfurther realized in the size and/or cost of hardware implementation ofimage processor section 1140B. An example of a filter kernel where thescalings between adjacent pixels are factors of powers of plus or minustwo is [3, −12, 6, 0, −3, 0].

FIG. 11C shows an embodiment of a distributive-arithmetic propertyfiltering operation. Optics 1101C, aspheric optical element 1110C,detector 1120C and data stream 1125C cooperate such that a filter kernel1130C utilizes distributive property of arithmetic filtering and imageprocessing section 1140C utilizes specialized distributive arithmetictaps 1145C. The tap 1145C performs accumulation of scaled values. Thereduced set filter kernel of filter kernel 1130C is such that thedistribution of all coefficients supports efficient implementation. Forexample, filter kernel 1130C and image processing section 1140C may beconfigured such that there are only two scale factors or multipliersthat are implemented, providing five distinct coefficients available forreconstruction (5 by considering positive and negative values of the twocoefficients, plus the zero coefficient). Scaling filter tap 1145Csupports efficient implementation because a single tap can be“overclocked” to service many pixels. Certain implementations may preferto overclock a multiplier as in 1145C, or some may to prefer tooverclock an accumulator as in 1145D, FIG. 11D. In prior art linearfiltering, such as employed by FIR tap delay lines, a separate anddistinct operator was required for each pixel and coefficient.

FIG. 11D shows an embodiment of a distributive-arithmetic propertyfiltering operation. Optics 1101D, aspheric optical element 1110D,detector 1120D and data stream 1125D cooperate such that a filter kernel1130D utilizes distributive-arithmetic filtering and image processingsection 1140D utilizes specialized distributive-arithmetic taps 1145D.The tap 1145D performs scaling of accumulated values. The reduced setfilter kernel of filter kernel 1130D is such that the distribution ofall coefficients supports efficient implementation. For example, filterkernel 1130D and image processing section 1140D may be configured suchthat there are only two scale factors or multipliers that areimplemented, providing 5 distinct coefficients available forreconstruction (5 by considering positive and negative values of the twocoefficients, plus the zero coefficient). As in FIG. 11C, scaling filtertap 1145D supports efficient implementation because a single tap can be“overclocked” to service many pixels. Certain implementations may toprefer to overclock an accumulator as in 1145D. In prior art linearfiltering, such as employed by FIR tap delay lines, a separate anddistinct operator was required for each pixel and coefficient.

One method of designing certain of the wavefront coded aspheric opticalelements (e.g., distinct elements 1110 of FIGS. 11A-D) and certain ofthe filter kernels (e.g., distinct kernels 1130 of FIGS. 11A-D) utilizesa plurality of weight matrices that target image processing. Othersystem design goals, such as image quality, sensitivity to aberrations,etc., may also have associated weight matrices. As part of the designand optimization process, the weight matrices may be optimally joined toachieve the selected goals. Since the optics and the signal processingcan be jointly optimized, many types of solutions are reachable that areotherwise difficult or impossible to recognize if, for example, only oneof the optics or signal processing is optimized.

By way of example, one weight matrix for a generalized integer filter(e.g., utilized by logic of FIG. 6A) has values of zero for all possibleinteger values. In contrast, a weight matrix for a power of two reducedset filter kernel (e.g., utilized by logic of FIG. 6B) has values ofzero only for integers that are a power of two. The integer zero has azero weight value since this value is trivial to implement. Otherintegers have higher weight values. When weight values are for integersthat are not a power of two, but are above zero, the optimizationbetween optics, filter kernel, image processing, and final image mayoccur to trade cost to performance by setting appropriate weight valuesabove zero.

In one example, in a reduced set filter kernel utilizing sums of powersof two, the weight matrix may have zero value for integers that are apower of two, a larger value for integers that are constructed from thesum or difference of two power of two values (such a 6=4+2 or 7=8−1),and still larger values for integers that are constructed from threepower of two values (such as 11=8+2+1, or 11=8+4−1). When both positiveand negative values of power of two are used, the weight for integerssuch as seven are reduced, since 7=8−1. By way of contrast, the generalimplementation (e.g., FIG. 6A) requires three summations, i.e., 7=4+2+1.

The weight matrix can also be used to optimize a gradient filter (e.g.,filter 1130A, FIG. 11A), a scaling filter (e.g., filter 1130B, FIG.11B), and distributive property kernels. Adjacent integer values may begiven a weight based on their difference or gradient, or scale factor.In one example, every pair of integer values is assigned a zero value iftheir difference or scale is a power of two or otherwise a large value.The pairing of coefficients also depends on the readout of pixels sincethe readout mode determines the adjacency rules for neighboring pixelsand since both the readout and kernel orientation are included in theweight matrix. Those skilled in the art appreciate that other gradientkernel filters may assign small values when the difference of theintegers is a sum of powers of two, and so on.

FIG. 12 illustratively shows a final image 1201 processed by system 100of FIG. 1 and a final image 1202 processed by system 200 of FIG. 2. Inthis example, both systems 100, 200 have the same optics 101, 201 anddetector 120, 200, respectively, but have different filter kernels andimage processing sections (implementing respective one dimensionalkernel filters within rectangularly separable processing). Image 1201was formed with a filter kernel 130 that is a generalized integerfilter. Image 1202 was formed with a filter kernel 230 that is a reducedset filter kernel utilizing power of two values. Both images 1201, 1202are similar, but image 1202 has improved contrast, as shown. Moreover,in comparison, reduced set filter kernel 230 has a smaller and lesscostly implementation then the generalized integer filter of filterkernel 130.

FIG. 13 illustratively shows a final image 1301 processed by system 100of FIG. 1 and a final image 1302 processed by system 200 of FIG. 2. Asin FIG. 12, both systems 100, 200 have equivalent optics and detectors101, 201 and 120, 220, respectively, but have different filter kernelsand image processing sections (implementing respective one dimensionalkernel filters within rectangularly separable processing). System 200employs a kernel filter 1130A, FIG. 11A, as kernel filter 230, FIG. 2.System 100 employs a generalized integer filter as kernel filter 130;thus image 1301 was formed with the generalized integer filter. Bothimages 1301, 1302 are similar, except that image 1302 has lowercontrast. Since contrast is also one possible imaging goal, the reducedcontrast can be set to meet this goal. Moreover, the reduced setgradient filter kernel used to process image 1302 can be smaller andless costly to implement then the generalized integer filter.

FIG. 14 shows the frequency response for the three digital filters usedin FIG. 12 and FIG. 13. Each filter has a slightly different frequencyresponse and a different implementation. Certain imaging applicationsprefer one frequency response while another may prefer a differentfrequency response. The particular reduced set power of two frequencyresponse (e.g., employed in filter kernel 230, FIG. 2) has the highestcontrast. The generalized integer frequency response has the nexthighest contrast, and the reduced set gradient power of two filterkernel 1130 has the lowest frequency response in this example. This neednot be the general case. Accordingly, depending on the application, theappropriate response is chosen; if the filter kernel also employs areduced set of filter kernel values, then the implementation may also bemade with reduced cost and complexity.

With further regard to FIG. 10, optimizing image processing for colormay have certain advantages. FIG. 15 thus illustrates one other opticalimaging system with optimized color image processing. System 1500includes optics 1501 and a wavefront coded aspheric optical element 1510that cooperate to form an intermediate image at a detector 1520. Element1510 operates to code the wavefront within system 1500. A data stream1525 from detector 1520 is then processed by a series of processingblocks 1522, 1524, 1530, 1540, 1552, 1554, 1560 to decode the wavefrontand produce a final image 1570. Block 1522 and block 1524 operate topreprocess data stream 1525 for noise reduction. In particular, FPNblock 1522 corrects for fixed pattern noise (e.g., pixel gain and bias,and nonlinearity in response) of detector 1510; pre-filtering block 1524utilizes knowledge of wavefront coded optical element 1510 to furtherreduce noise from data stream 1525. Color conversion block 1530 convertsRGB color components (from data stream 1525) to a new colorspace. Blur &filtering block 1540 removes blur from the new colorspace images byfiltering one or more of the new colorspace channels. Block 1552 andblock 1554 operate to post-process data from block 1540, for furthernoise reduction. In particular, SC Block 1552 filters noise within eachsingle channel of data using knowledge of digital filtering within block1540; MC Block 1554 filters noise from multiple channels of data usingknowledge of digital filtering within block 1540. Prior to image 1570,another color conversion block converts the colorspace image componentsback to RGB color components.

FIG. 16A schematically illustrates one color processing system 1600 thatproduces a final three-color image 1660 from a color filter arraydetector 1602. System 1600 employs optics 1601 (with one or morewavefront coded optical elements or surfaces) to code the wavefront ofsystem 1600 and to produce an intermediate optical image at detector1602. Wavefront coding of optics 1601 thus forms a blurred image ondetector 1602. This intermediate image is then processed by NRP andcolorspace conversion block 1620. The noise reduction processing (NRP)of block 1620 for example functions to remove detector non-linearity andadditive noise; while the colorspace conversion of block 1620 functionsto remove spatial correlation between composite images to reduce theamount of silicon and/or memory required for blur removal processing (inblocks 1642, 1644). Data from block 1620 is also split into twochannels: a spatial image channel 1632 and one or more color channels1634. Spatial image channel 1632 has more spatial detail than colorchannels 1634. Accordingly, the dominant spatial channel 1632 has themajority of blur removal within process block 1642. The color channelshave substantially less blur removal processing within process block1644. After blur removal processes, channels 1632 and 1634 are againcombined and are processed within noise reduction processing andcolorspace conversion block 1650. This further removes image noiseaccentuated by blur removal; and colorspace conversion transforms imageinto RGB for final image 1660.

FIG. 17 through FIG. 23 further illustrate colorspace conversion andblur removal processes for a particular embodiment of system 1600.Detector 1602 is a detector with a Bayer color filter array. FIG. 17shows red, green and blue component images 1700, 1702 and 1704,respectively, of an actual object imaged through system 1600 but withoutwavefront coding in optics 1601 (i.e., the wavefront coded optics arenot present with optics 1601). In images 1700, 1702, 1704, it isapparent that each color image has a high degree of spatial similarityto each other image, and that many parts of the image are blurred. Theblurred images are not correctable without wavefront coding, asdescribed below.

FIG. 18 shows raw red, green and blue component images 1800, 1802 and1804, respectively, of the same object imaged through system 1600 withwavefront coded optics 1601. Images 1800, 1802 and 1804 represent imagesderived from data stream 1625 and prior to processing by block 1620.Each component image 1800, 1802, 1804 is again fairly similar to eachother image; and each image is blurred due to aspheric optics 1601.

FIG. 19 shows the color component images of FIG. 18 after blur removalprocessing of block 1642; image 1900 is image 1800 after block 1642;image 1902 is image 1802 after block 1644; and image 1904 is image 1804after block 1644. For this example, processing block 1644 performed noprocessing. Again, processing blocks 1620 and 1650 were not used. Eachcomponent image 1900, 1902 and 1904 is sharp and clear; and thecombination of all three component images results in a sharp and clearthree color image.

FIG. 20 shows component images 2000, 2002, 2004 after colorspaceconversion (block 1620) from RGB to YIQ colorspace, before processing ofspatial and color channels 1632 and 1634. Component images 2000, 2002,2004 thus represent component images of FIG. 18 after processing byblock 1620, on channel 1632 and channel 1634. Notice that Y channelimage 2000 is similar to the red and green images 1800, 1802 of FIG. 18.The I and Q channel images 2002, 2004 are however much different fromall channels of FIG. 18. The YIQ colorspace conversion (block 1620)resulted in the Y channel component image 2000 containing the majorityof the spatial information. The I channel component image 2002 containsmuch less spatial information than does the Y channel, although somespatial information is visible. Notice the differences in the intensityscales to the right of each image. The Q channel component image 2004has very little spatial information at the intensity scale shown forthese images.

FIG. 21 shows the component YIQ wavefront coded images 2100, 2102, 2104after blur removal process 1642 of the spatial channel 1632. The I and Qchannel images 2002, 2004 were not filtered in block 1644, therebysaving processing and memory space; accordingly, images 2102, 2104 areidentical to images 2002, 2004. Those skilled in the art appreciate thatimages 2002, 2004 could be filtered in block 1644 with low resolutionand small bit depth filters. After the filtered YIQ image of FIG. 21 istransformed back into RGB space (block 1650), the final three colorimage 1660 is sharp and clear with much less processing then required toproduce the final image based on FIG. 19.

The YIQ colorspace is one of many linear and non-linear colorspaceconversions available. One method for performing colorspace conversionis to select a single space to represent the entire image. Wavefrontcoded optics and processors can cooperate such that an optimal, dynamic,colorspace conversion can be achieved that reduces the amount of signalprocessing and improves image quality. In FIG. 16A, for example,wavefront coded system 1600 may employ a colorspace process block 1620that varies spatially across the entire image. Such spatial variationmay be denoted as Dynamic Color Space (DCS). This spatially-varyingcolorspace conversion 1620 performs a single transformation on a region,where the region is dynamic and not constrained or enclosed. Forexample, an image with blue sky in the upper half and sandy beach in thelower half is well suited to the following split colorspace: a“blue-spatial-information-preserving” colorspace conversion for theupper half of the image and a “speckle-brown-information-preserving”colorspace transformation for the lower half. Such spatially varyingregions can be defined as an ordered set of rectilinear blocks, or asrandomly assigned pixels, or as dynamically allocated contours that aredefined in an optimal fashion as each scene changes. Dynamic blockallocation methods in conversion process 1620 can be sensitive to thetotal computational power and desired throughput and accuracy tradeoffsdesired in the imaging system, and can be guided or limited by knowledgeof the spatial correlating effects of optics 1601.

In general, DCS can provide combined noise reduction, color-plane arrayinterpolation, and reduced-cost image deblurring. The DCS algorithm isappropriate for a variety of processing approaches within software andhardware based systems. One version of DCS implements dynamic linearcolorspace conversion. Other versions of DCS implement dynamicnon-linear colorspaces. DCS may be combined or preceded by non-linearcolorspace conversions or other transformations, such as HSV, dependingon processing system resources and application goal.

The general DCS algorithm can be understood through correlation andprinciple component analysis of FIG. 16B. FIG. 16B fits entirely withinthe block 1620 in FIG. 16A. The Bayer sensor 1602 and the output image1632 are numbered identically to FIG. 16A to highlight the connection.Since DCS can be a reversible transform, those skilled in the artrecognize that an inverse, either exact or approximate, DCS processoccurs entirely within block 1650 of FIG. 16A. The determination of theprinciple components (or principal color or colors) from the estimatedcorrelation of colors in multi-channel images of an arbitrary region ofpixels with an arbitrary number of color channels (more than one)determines the optimal (in the least-squares sense) colorspacetransformation for that particular region. In other words, DCS allowsblocks 1620 (and 1650 via the inverse DCS transformation) in FIG. 16A tobe optimized jointly with all other wavefront coding system parameters.In the Bayer detector case with detector 1602 in FIG. 16B, a colorchannel matrix is formed by collecting the sampled color channels intocolumns of a matrix 1603A. Various schemes can be used to provideestimates of image color channels at each desired location. Not allpixel locations need to be estimated for the entire region. As anexample, FIG. 16B shows a zero-order hold method that essentially copiesnearby red and blue pixel values to every green pixel location, andgenerates 8 rows of data in the 8×3 matrix, from 16 pixels in the 4×4region. A principle component decomposition of the correlation estimate1603B of the color channel matrix 1603A provides the colorspacetransformation 1603C. This is only one such example stacking of valuesto form a color channel matrix for multi-channel color systems; thoseskilled in the art appreciate that other arrangements are possiblewithout departing from the scope hereof.

An estimate of the principle components 1603C of the correlation matrix1603B forms the optimal colorspace transformation matrix for thisparticular region and channel selection. Obtaining the spatial intensityimage 1632 in FIG. 16A is achieved by applying the first principlecomponent to the color channel matrix 1603A in FIG. 16B, forming thetransformed spatial image 1632 in FIG. 16A. The deblurring function 1642in FIG. 16A then operates on the spatial image 1632, providing imagereconstruction and simultaneous Bayer-pattern interpolation. Returningto the original colorspace then occurs in block 1650 by re-transformingthe reconstructed spatial and color channels using a form of an inverseof the color channel transformation matrices 1603C determined for eachregion, producing the final image 1660. Color channel images 1634 mayalso be extracted from DCS by using the secondary and tertiary principlecomponents in this example.

Upon DCS transformation, it is not however guaranteed that only thespatial channel 1632 is to be processed by the spatial channel blurremoval function 1642. Each block or region that was transformedcontains it's own unique mapping matrix but all regions share a commonspatial image 1642. In many cases, the optimal transformation for agiven region does not achieve complete transformation of all spatialinformation to the spatial image 1642. In such a case, deblurring of thecolor channels may be made by blur removal process block 1644. Finalimage assembly is then performed by “coloring” the dynamic regions usingrespective matrices of colorspace inversion 1650.

FIG. 22 shows the component images of FIG. 18 after colorspaceconversion with the DCS algorithm. After conversion with the dynamicallychanging colorspace, essentially all spatial information is contained inchannel 1 (2200). The other two color channels 2 (2202), 3 (2204) haveessentially no spatial information relative to the shown intensityscale, such that they may be removed from the final image for noisereduction. FIG. 23 shows these component images after filtering toremove image blur. Only channel 1 (2300) was filtered in this example,again due to low resolution in channels 2 (2302), 3 (2303). Afterconversion from DCS to RGB (process block 1650), the final image isagain sharp and clear, and with fewer color artifacts as compared to thefinal image of FIG. 21 (and with less stringent processing and memoryrequirements).

The wavefront coded imaging systems discussed herein may have certainadvantages when used within electronic devices such as cell phones,video conferencing apparatus and miniature cameras. FIG. 24 shows onesuch electronic device 2400 to illustrate such advantages. Device 2400has an optical lens 2402 that forms an image from object space 2404 ontoa digital detector 2406 (e.g., a 3-color CMOS array), as shown. Detector2406 converts the image into a data stream 2408. A microprocessor 2410processes data stream to generate a final image 2412.

Optical lens 2402 is also wavefront coded; one surface 2414 of lens 2402is for example a wavefront coded aspheric optical element. Accordingly,optical lens 2402 and element 2404 may function as the optics andwavefront coded elements discussed hereinabove (e.g., optics 201,element 210 of FIG. 2). Microprocessor 2410 is for example a processingsection discussed hereinabove (e.g., image processing section 240, FIG.2). Microprocessor 2410 employs a filter kernel in processing datastream 2408, thereby decoding the wavefront due to phase manipulation bylens 2402 and generating a crisp image 2412. Image 2412 may for examplebe displayed on an LCD display 2415 or other display screen.

The effects of wavefront coding of optical lens 2402 and post-processingby microprocessor 2410 are for example constructed and arranged toreduce mis-focus related aberrations such as defocus, field curvature,or manufacturing and assembly related mis-focus. Accordingly, electronicdevice 2400 may employ a single (plastic) optical element 2402 withoutthe need for other complex optical elements.

Moreover, by selecting the appropriate filter kernel (e.g., a reducedset filter kernel with power of two kernel values) and the correspondingphase mask (i.e., surface wavefront coded element 2414), microprocessor2410 may operate with reduced processor load and memory requirement. Inone example, microprocessor 2410 employs image processing section 540and kernel 530, FIG. 5, to reduce such processor load and memoryrequirement. In another example, microprocessor 2410 employs logicarchitectures of FIG. 6B and/or FIG. 7 to reduce such processor load andmemory requirement. Likewise, microprocessor 2410 may employ imageprocessing section 840 and kernel 830 to achieve certain otheradvantages, as discussed in FIG. 8. Depending upon designconsiderations, including color processing goals, microprocessor 2410may for example employ processing techniques disclosed in connectionwith FIG. 10, FIG. 16A and/or FIG. 16B. In another example, electronicdevice 2400 may employ opto-electronic components that implement imagingarchitectures of FIG. 9A or FIG. 9B, or one of FIG. 11A-11D. By reducingprocessing requirements, electronic device 2400 may be constructed witha less expensive processor or with less memory. Those skilled in the artappreciate that memory savings may similarly translate to other memorydevices or cards 2416, resulting in further savings.

As those in the art appreciate, electronic device 2400 may furtherinclude other electronics 2420 to incorporate other desired operationand functionality, for example cell phone functionality.

It is therefore apparent that certain trade-offs or “optimizations” maybe made within the above described wavefront coded optical imagingsystems, to achieve image characteristic goals and/or cost goals, forexample. By way of example, consider FIG. 25, illustrating components ina wavefront coded imaging system. Component optics 2501 is for exampleoptics 201 FIG. 2. Phase mask 2410 is for example wavefront codedaspheric optical element 210, FIG. 2. Detector component 2520 is forexample detector 220, FIG. 2, while kernel filter 2530 is for examplefilter kernel 230, FIG. 2. Image processing section 2540 is for examplesection 240, FIG. 2.

FIG. 25 illustrates one three-component optimization 2570 utilizing areduced set filter kernel 2530 (e.g., FIG. 8); optimization 2570 thusties together (in design) wavefront coded optics 2510, filter kernel2530 and image processing section 2540. A standard detector 2520 andoptics 2501 is not necessarily optimized with optimization 2570, ifdesired. In another example, one four-component optimization 2572utilizes a detector 2520 with a particular readout format (e.g., as inFIGS. 9A, 9B). In optimization 2572, therefore, wavefront coded optics2510, filter kernel 2530 and image processing section 2540 are also tiedtogether (in design). In an example optimizing color images (e.g., as inFIG. 10), one two-component optimization 2574 can include acolor-specific detector 2520 tied together (in design) with imageprocessing section 2540. As those in the art appreciate, optics 2501 maybe optimized with any of optimizations 2570, 2572, 2574 to achieveimaging characteristics that support other optimizations, for example.

In certain embodiments herein, the form of optics and optical elementsused in the above-described imaging systems are also optimized. Suchform is for example to provide low variability to misfocus-likeaberrations, high MTFs, and low noise gain values. As described in moredetail below, these optics and optical elements may make up, or be partof, imaging systems such as microscopes, endoscopes, telescopes, machinevision systems, miniature cameras, and cell phone cameras, videocameras, digital cameras, barcode scanners, biometrics systems, etc. Twoexample, easy to describe, optical forms of wavefront coded optics andsuitable for system optimization herein may include weighted sums ofseparable powers, p(x,y)=Sum ai[sign(x)|x|^bi+sign(y)|y|^bi], and thecosinusoidal forms p(r,theta)=Sum ai r^bi*cos(ci*theta+phii). Asdescribed in more detail below, the optics also may include specializedcontour surfaces (sometimes denoted herein as “constant profile pathoptics”) that provide phase variation within the wavefront coded opticalelement (e.g., element 210, FIG. 2). Again, the wavefront coded opticalelement and optics (e.g., element 210 and optics 201, respectively) maybe combined as a single optical element or system (e.g., utilizinglenses and/or mirrors). In one example, the optics produce an MTF withinthe optical imaging system with a frequency domain that is complimentaryto the kernel filter; for example, high power within the MTF implies lowpower in filter kernel. The MTF and filter kernel may provide a degreeof spatial correlation (e.g., between the filter kernel and wavefrontcoded aspheric optical element) such as described herein.

As noted above, these wavefront coded imaging systems havenon-traditional aspheric optics and image processing. One possible goalfor such systems is to produce a final image that is substantiallyinsensitive to misfocus-like aberrations. This insensitivity supports(a) a large depth of field or depth of focus, (b) tolerance tofabrication and/or assembly errors that for example create misfocus-likeaberrations, and/or (c) optically-generated misfocus-like aberrations(e.g., spherical aberrations, astigmatism, petzval curvature, chromaticaberration, temperature-related misfocus). Internal optics within suchimaging systems form images with specialized blur that may also beinsensitive to these misfocus-like aberrations. Moreover, the effects ofcoma can also be reduced within such wavefront coded optical imagingsystems. After capture by a detector (e.g., a digital detector 220capturing the intermediate image to generate a digital data stream 225,FIG. 2), image processing may operate to remove the blur associated withthe intermediate image to produce a final image that is sharp and clear,and with a high signal to noise ratio (SNR).

The design process to enable such wavefront coded imaging systems may besuch that the optical system is insensitive to misfocus-likeaberrations, the size and shape of the point spread function (PSF) iscompact and constant as a function wavelength, field angle, objectposition, etc, and the resulting MTF has high values. The followingconstant profile path optics provide efficient designs for use in suchsystems.

Certain constant profile path optics are based on parametricallyefficient descriptions (e.g., low number of parameters) of optical form;such form may for example support high performance optical imagingsystems with selected imaging and/or cost characteristics. In general,these parametrically efficient forms are not required. That is, thesurface height of each part of the aspheric surface can be designated asan independent variable used in design and optimization; however thenumber of variables to optimize this general case is extremely large andimpractical. Constant profile path optics thus facilitates a powerfuland general optical form for use in design and optimization.

In one embodiment, constant profile path optics include aspheric opticalelements where the surface height is defined along paths and where thefunctional form, or profile, of the surface is the same along normalizedversion of the paths. The actual surface height varies from path topath, but the functional form or profile along each path does not. Suchsurface profiles on an optical element operate to modify phase of awavefront within the optical system; image effects caused by these phasechanges is then reversed in image processing, e.g., through operation ofthe filter kernel. By way of example, when certain constant profile pathoptics have modified wavefront phase within the optical imaging system,the resulting MTF at the intermediate image (captured by a detector)correlates to the functional and spatial characteristics of the filterkernel used in image processing. The constant profile path optics andthe filter kernel may therefore be complimentary to one another toprovide the desired imaging characteristics.

Four examples 2600A-2600D of constant profile path optics are shown inFIG. 26. Consider for example the paths within profile 2600A. Thesepaths are along square contours of the optics; for this optic form, thenormalized surface height is the same over normalized versions of thesquare contours. The paths of profile 2600B are pentagon-shaped; thefunctional form of the surface height along a normalized version of eachpentagon is the same. The paths of profiles 2600C and 2600D are cross-and star-shaped, respectively. Similar to profiles 2600A, 2600B, suchfunctional forms have common normalized surface heights along commonpaths.

FIG. 27 show other variations 2700A, 2700B in constant profile pathoptics Profile 2700B represents wavefront coded optics where the pathstraces can have closed contours, such as shown by region #1. The pathsin profile 2700A on the other hand trace open contours. In bothprofiles, region #1 contains a set of related paths. The functional formof the surface height along a normalized version of each path in region#1 is the same. The same is true for regions 2, 3, 4, and 5. The actualsurface height along each path within each region can be different. Thefunctional forms over different regions can be related or not. Theoptics of profile 2700B shows a combination of paths that are open andclosed; the paths of region #1 trace closed contours while the paths ofthe other four regions trace open contours. Accordingly, the functionalform of the surface height along each path may be the same for eachregion, yet the actual surface height can change within the region.

As those skilled in the art will appreciate, one of several variationsof the path profiles shown in FIG. 26 and FIG. 27 may includenon-straight line paths. That is, straight line paths such as shown inprofiles of FIG. 26 and FIG. 27 may take other forms wherein the pathsare composed of curved segments. For example, the pentagon shaped formof profile 2700A split into straight line regions may instead be ofcircular form with separate regions defining arcs as the paths.

The type of path traced in constant profile path optics can also changeover different regions, as in FIG. 28. The paths in the outer region ofprofile 2800A trace closed square contours, while its inner region pathstrace closed circular contours. These inner and outer regions can bereversed, as shown by profile 2800B. Only some of the paths in the outerregion can be closed, as shown in 2800B.

FIG. 29 shows two profiles 2900A, 2900B where at least one region of theoptics is not altered with constant profile path optics. In profiles2900A, 2900B, the paths form contours only in the outer region of theoptics; the inner region has no paths and thus has no specializedsurface shape. The optics of profiles 2900A, 2900B may be thought of asthe optics associated with profiles 2600A, 2600B, respectively, but withan amplitude of zero applied to the paths in the inner region.

The parameters that describe the specialized surfaces of FIG. 26-FIG. 29may be considered as a composition of two components: 1) the parametersthat describe the profile of the optical surface height along the pathsof the particular optic and 2) the parameters that describe the surfaceheight across the paths. The second set of components may for exampledescribe an amplitude change for each path, and between each path, in agiven region. If the amplitude of the set of paths in a region is set tozero, then optics as in FIG. 29 may result.

In one embodiment, the mathematical description of a surface of oneregion of a constant profile path optic is described as:S(R,theta, a,b )=C( a )D( b )where the optical surface along each path in the region is parameterizedby C(a), and evaluated at D(b) for the particular path in the region.Parameters ‘a’ and ‘b’ define the characteristics of the particularsurface. The contributions from C(a) are constant over all paths in aregion. The contributions from D(b) change for each path in a region.Parameters C(a) may define the surface along each path and the overallsurface modulated between or across the paths in a region of D(b). Theoptical surface of a constant profile path optics may also be separablein terms along the paths and across the paths for the particular optic.

Consider the optical surface of 2600A, where the paths define open sidesof square contours. This leads to four regions, the left side, rightside, top side, and bottom side. One example of a mathematicaldescription of the surface profile along the paths of the four regionsis:C( a )=a0+a1x+a2x2+ . . . |x|<1Thus a set of parameters ai form a polynomial description of the surfaceheight along each normalized path of the four regions of the optics. Inthis case, the length of each path in the regions is normalized to unityso that a “stretching” of the fundamental surface profile C(a) isapplied to each path. The path length does not have to be considered anormalized length although it may be useful to do so.

Each path can be modified by a gain or constant term without alteringthe functional form of the surface along each path. One example of thegain applied across the paths in a region can be mathematicallydescribed asD( b )=b0+b1(PathNumber)+b2(PathNumber)2+b3(PathNumber)3+ . . .PathNumber=0, 1, 2, 3, . . .where the PathNumber parameter is a sequence of values that describesthe set of paths in a region. The number of paths in a region may belarge forming an essentially continuous surface. For example, the pathof region #1 (profile 2700A) closest to the optical center can beassigned PathNumber=0, the adjacent path slightly farther from theoptical center can be assigned PathNumber=1, the next adjacent path canbe assigned PathNumber=2, and so on. The overall surface is thenmathematically composed of the product of the “along the paths” surfacedescription and the “across the paths” surface description.Mathematically, the surface description in one region may be given by:S(R,theta, a,b )=C( a )D( b ), where the optical surface is nowparameterized by the parameter vectors a and b.

If the functional form of the surface along the paths is a polynomial ofsecond order, or less, then the second derivative along the normalizedpaths is a constant. In this special case, the constant profile pathoptics may for example be denoted as “constant power path optics.”Constant power path optics are particularly effective forms for thewavefront coded optics. In terms of optical parameters, a second orderpolynomial C(a) can be described as:C( a )=thickness+tilt+optical power,where the zeroth order term a₀ describes the amount of thickness, thefirst order term a₁x describes the amount of tilt, and the second orderterm a₂x² describes the amount of optical power. If higher order termsare used, then the amount of optical power, or second derivative, canchange along the contour. Higher order terms can also more accuratelymodel optical power since a second order polynomial is an approximationto a sphere.

For certain optical imaging systems, such higher order terms can beimportant. If the tilt parameter is non-zero, then optical surfaces withdiscontinuities may be described. Given the high degree of accuracy thatfree-form surfaces can be fabricated today, surfaces withdiscontinuities need not be avoided. For certain optical systems, theacross-the-paths term D is chosen so that the central region of theoptical surface is flat (or nearly flat). Often the slope of D canbecome quite large near the edge of the surface. The central region mayalso process optical imaging rays that need not change in order tocontrol misfocus aberrations, as rays outside of the central regiontypically cause the misfocus effects within imaging systems.

Since central rays contribute less to misfocus effects then the outerrays, the shape of the contours can change for the central and outerregions to affect these rays differently. This allows shaping of the in-and out-of-focus PSFs, high MTFs, and customization of the powerspectrum for digital filtering. The power spectrum of the digital filtermodifies the power spectrum of the additive noise after filtering. Noisereduction techniques after filtering are thus increasingly effective byjointly designing the optics and digital processing for noise reductionand imaging, such as described above.

Examples of Constant Profile Path Optics

Certain constant profile path optics examples described below follow theform S(R,theta,a,b)=C(a)D(b) where the particular paths used are thoseof profile 2600A, with each side of the square contour defining one offour regions. FIG. 30 shows one surface profile 3000, its in-focus MTF3002, and along the path surface form C(a) 3004 and across the pathamplitude D(b) 3006. A third order polynomial has been used for acrossthe path amplitude 3006 and a second order polynomial has been used forthe along the path surface form 3004. This type of optics has anapproximately constant amount of optical power along the normalizedsquare path contours. The surface profile 3000 and in-focus MTF 3002have been drawn via contours of constant surface and MTF height. Noticethat the constant height contours vary from roughly circular torectangular along the diameter of the surface. The resulting in-focusMTF has non-circular contours that roughly follow a “+” shape. Thefunctional form in waves along the paths and across the path amplitudeis shown in the bottom graphs 3004, 3006. About one wave of opticalpower is used along the sides of the paths (as shown by the minimum of2.8 and a max of 3.8 waves) with an amplitude variation between −0.1 and−0.4 across the paths.

Another example of constant profile path optics is shown in FIG. 31. Thepolynomial form S(R,theta,a,b)=C(a)D(b) is used, with C(a) being asecond order constant optical power polynomial and D(b) being a thirdorder polynomial. Notice that the surface profile 3100 and MTF 3102 arevery different from those of FIG. 30. The MTF 3102 for this example ismore compact then MTF 3002 of FIG. 30, with asymmetry also lesspronounced. About six waves of optical power are used along the sides ofthe path contours as shown in 3104, with an increasing amplitude usedacross the paths having a maximum value of about 4.

FIG. 32 shows another example that follows the form of FIG. 30 and FIG.31 with a second order constant optical power polynomial describing thefunctional form along the paths and a third order polynomial describingthe across the path amplitude; except the form of the amplitude functionchanges to allow a flat center region. The surface profile 3200 has afour-sided nature with a flat center, while its resulting MTF 3202 isroughly a four sided pyramid. Such an MTF shape 3200 is suitable fornon-uniform sampling, such as found on the green channel of three colorBayer color filter array detectors (see FIGS. 16-23). Less then one waveof optical power is used along the paths 3204 with an amplitudevariation from 0 to −9 used across the paths 3206.

FIG. 33 shows one example of a constant profile path optics withpentagon-shaped paths as in profile 2700A, FIG. 27. For this example,the functional form of the surface is the same for each straight linesegment within the five open pentagon path regions. A second orderpolynomial describes the surface along the paths (3304) while a thirdorder polynomial describes the amplitude across the paths (3306). Theoptical surface (profile 3300) has ten unequally shaped lobes and afairly flat central region. The corresponding MTF 3302 has near-circularform about the center of the 2D MTF plane. About 1.5 waves of opticalpower is used along the paths 3304 with an amplitude of zero to 18 usedacross the paths 3306.

FIG. 34 shows the sampled PSFs from the example of FIG. 32 withoutwavefront coding, and for a range of defocus values. The small squaresin each mesh drawing of each PSF represent individual pixel samples(output from the detector). The physical parameters used to generatethese PSFs are: 10 micron illumination wavelength; grayscale pixels with25.4 micron centers and 100% fill factor; working F-number of the opticsis 1.24; misfocus aberration coefficient W₂₀ varies from 0 to 2 waves.Accordingly, it is apparent that without wavefront coding, the sampledPSFs have an unavoidable large change in size due to misfocus effects.

FIG. 35 similarly shows the sampled PSFs from the example of FIG. 32 butwith wavefront coding. Notice that the sampled PSFs (for the range ofdefocus values) have a sharp spike and a broad pedestal that decreaseswith the distance from the sharp spike. Each of the PSFs is similar andessentially independent of misfocus. The small squares in the meshdrawing again represent individual pixels.

FIG. 36 and FIG. 37 show and compare cross sections through sampled PSFsfrom FIG. 34 and FIG. 35. The PSFs of FIG. 36 and FIG. 37 are shown withfive misfocus values evenly spaced between 0 and 2 waves. The sampledPSFs in FIG. 36 are normalized to constant volume, while the PSFs inFIG. 37 are normalized for a constant peak value. FIGS. 36 and 37 thusillustrate changes in the sampled PSFs of the system without wavefrontcoding and the lack of change in the sampled PSFs with wavefront coding.

FIG. 38 shows an example of one 2D digital filter (plotted as an imageand based on zero misfocus PSF) that may be used to remove the wavefrontcoding blur from the sampled PSFs of FIG. 35. This digital filter is forexample used within image processing (e.g., within section 240, FIG. 2)as a filter kernel. Notice that this kernel has high values only nearthe center of the filter and decreasing values related to the distancefrom the center. The filter, and the sampled PSFs of FIG. 35, arespatially compact.

After using the 2D filter of FIG. 38 on the sampled PSFs of FIG. 35, thePSFs of FIG. 39 result. Notice that these PSFs (representing PSFs withwavefront coding and after filtering within an image processing section)have nearly ideal form and are essentially unchanged over the range ofmisfocus.

FIG. 40 illustrates an amount of power contained in a geometric conceptknown as “rank” for the in-focus sampled PSF and the digital filter ofFIG. 35 and FIG. 38, respectively. Rectangularly separable optics, suchas a cubic described by P(x)P(y)=exp(j[X^3+Y^3]), forms a sampled PSFthat is closely approximated as p(x)p(y), where the independentvariables x and y are horizontal and vertical axes defined on a squaregrid. This separable nature allows rectangularly separable filteringthat is computationally efficient. One drawback of rectangularlyseparable optics is that the PSF can spatially shift with misfocus. Oneuseful feature of certain constant profile path optics is that they donot cause the PSF to spatially shift. Constant profile path optics mayalso produce PSFs, MTFs and corresponding digital filters thatapproximate low rank, to further facilitate efficient processing. Thetop plot of FIG. 39 shows that there are only two geometric ranks thathave appreciable value for this particular sampled PSF; this system isthus approximated by a rank two system. The corresponding digital filterfor this example (the lower plot of FIG. 39) is also low rank and can bemade at least as low as the sampled PSF. In practice, the filtering ofthis example PSF may be accomplished with computationally efficientseparable filtering.

FIG. 41 shows the corresponding MTFs before and after filtering and inthe spatial frequency domain. For comparison purposes, also shown arethe MTFs of the identical physical system with no wavefront coding. Bothsystems are shown with misfocus values from 0 to 2 waves in five steps.The MTFs of the system with no wavefront coding is seen to drasticallychange with misfocus. The MTFs of the system with the constant profilepath optics before filtering has essentially no change with misfocus.The filtered MTFs result from filtering by the 2D digital filter of FIG.38; such MTFs have high values and degrade only for the largest misfocusvalue. As appreciated by those skilled in the art, other forms ofconstant profile path optics may control the MTF profile over largeramounts of misfocus by utilizing more processing capability and/or lowerMTFs before filtering.

The mathematical form of the contour optics of FIG. 32 may be asfollows. The polynomial description of functional form along the pathsin four regions:C(x)=0.4661−0.7013x^2, |x|<1The polynomial description of across the path amplitude is:D(y)=−1.8182+0.5170y+2.520y^2−10.1659y^3, 0.25<y<1=−1.8182, 0<y<0.25.In this example, the form along the path contours is given by an evensecond-order polynomial; and the amplitude across the paths is given bya third order polynomial that changes form so that the central region isrelatively flat. Using higher order polynomials for along the paths andfor the across the path amplitude supports higher performance then shownin these examples.

Constant profile path optics may be designed with specialized techniquesthat allow optimization with non-ideal optics that can have largeamounts of aberrations, vignetting, and loose tolerances. Thesetechniques allow optical system design that is not practical bytraditional analytical methods. FIG. 42 illustrates a design process4200 that illustrates certain of these techniques.

In process 4200, optics are modified (in design) and the loop mayrepeat, as shown. For example, process 4200 includes step 4202 to designconstant profile path optics. With a model for the optical surfaces(step 4202), the effects of the lens aberrations can be added (step4204) with information about the digital detector (step 4206) in orderto accurately simulate the sampled PSF and MTF. These lens aberrationsare in general a function of field angle, wavelength, object positionand zoom position; the aberrations can also be a function of theparticular surface form of the constant profile path optic (step 4202).One aberration considered in design process step 4204 is vignetting.While vignetting is often used in design of traditional optics to tradeoff light gathering for sharpness, vignetting within wavefront codingoptics, can lead to a poor match between optical design and the actualoptical system. The optical surfaces and aberrations that lead tosampled PSFs can be either simulated through ray-based methods orFourier Transform methods depending on the speed of the lens beingdesigned and the spatial resolution of the digital detector. Both ofthese general types of PSF simulation methods are well known to thoseskilled in the art of optical simulation.

After the sampled PSFs and MTFs have been simulated (steps 4204, 4206),a digital filter is used (step 4208) to remove wavefront coding blur.This digital filter can be general and calculated for each iteration indesign process 4200 or can be fixed or limited in form. An example of alimited form digital filter is a rectangularly separable filter. If thedigital filter is limited in this way, the design of the constantprofile path optics may be optimized for minimum rank PSFs and MTFssince the rank of the separable filter is 1. Other limited digitalfilters are filters where costs are assigned to particular filter valuesand sequences of filter values in order to optimize the implementation(e.g., hardware) of the image processing section. These costs may bereduced or increased as part of design process 4200. Further examples oflimited digital filters are those that have particular power spectrumcharacteristics. Since the additive noise is modified by the powerspectrum of the digital filter, controlling the characteristics of thefilter controls the characteristics of the additive noise after digitalfiltering. Noise Reduction techniques can be optimized jointly with thecharacteristics of the noise by limited the digital filter.

After the sampled PSFs/MTFs have been filtered (step 4208), a qualityassessment is performed (step 4210). Assessment 4210 is typically systemand application specific but may include (a) the quality of the filteredPSFs/MTFs both within and outside of the design range and/or (b) thecharacteristics of the digital filter (and/or its implementation and/ornoise effects). The quality assessment is then used with non-linearoptimization to modify the optics and repeat the iteration throughprocess 4200. The modified optics can include particular surfaces thatcontain the wavefront coding as well as other surfaces, and/or thicknessand distances of elements within the optical system. The parameters ofthe functional form along the paths and across the path amplitude can besimilarly optimized.

Changes may be made in the above methods and systems without departingfrom the scope hereof. It should thus be noted that that the mattercontained in the above description or shown in the accompanying drawingsshould be interpreted as illustrative and not in a limiting sense. Forexample, those skilled in the art should appreciate that although thewavefront coded element is often shown separate from the optics withinan imaging system (e.g., element 210 separate from optics 201), thesecomponents may be combined as a single item or group of items withoutdeparting from the scope hereof, for example as shown in FIG. 24. Thefollowing claims are intended to cover all generic and specific featuresdescribed herein, as well as all statements of the scope of the presentmethod and system, which, as a matter of language, might be said to fallthere between.

1. An image processing method, comprising the steps of: wavefront codinga wavefront that forms an optical image; converting the optical image toa data stream; and processing the data stream with a reduced set filterkernel to reverse effects of wavefront coding and generate a finalimage, the reduced set filter kernel having coefficients consisting ofpower of two coefficients.
 2. The image processing method of claim 1,the step of processing comprising the step of utilizing a filter kernelthat is complementary to a spatial frequency spectrum of the opticalimage.
 3. The image processing method of claim 2, the steps of wavefrontcoding, converting and processing occurring such that the spatialfrequency spectrum is spatially correlated to mathematical processing ofthe data stream with the reduced set filter kernel.
 4. The method ofclaim 1, the step of wavefront coding comprising the step of utilizingan aspheric wavefront coded optical element.
 5. The method of claim 1,the step of wavefront coding comprising wavefront coding the wavefrontwith constant profile path optics.
 6. The method of claim 5, furthercomprising the step of formulating the reduced set filter kernel to aspatial frequency spectrum of the optical image resulting from phasemodification of the wavefront through the constant profile path optics.7. The method of claim 1, the step of processing comprising the step ofprocessing the image, for each pixel, with filter tap logic consistingof a shifter and an adder.
 8. The method of claim 1, the step ofprocessing comprising the step of processing the data stream with areduced set filter kernel consisting of a plurality of regions whereinat least one of the regions has zero values.
 9. The method of claim 8,the step of wavefront coding comprising the step of wavefront coding thewavefront such that a Point Spread Function (PSF) spatially correlatesto the regions of the reduced set filter kernel.
 10. The method of claim1, the step of converting the optical image to a data stream comprisingoutputting data from a detector in a non-rectilinear format.
 11. Themethod of claim 10, the format comprising one of a diagonal readout anda circular readout.
 12. The method of claim 10, the step of processingcomprising processing the data stream with a reduced set filter kernelmatched to the non-rectilinear format.
 13. The method of claim 12,further comprising the step of remapping data to a rectilinear format.14. The method of claim 10, the step of wavefront coding comprisingwavefront coding the wavefront such that the optical image at thedetector substantially coincides with the non-rectilinear format of thedetector.
 15. The method of claim 1, the step of processing comprisingthe steps of utilizing a reduced set scaling filter kernel and a seriesof scaling taps.
 16. The method of claim 15, the step of processingcomprising the step of processing with a single summation for anycoefficient value of the filter kernel.
 17. The method of claim 15, thestep of wavefront coding comprising the step of wavefront coding thewavefront such that a PSF substantially contains information within aspatial pattern corresponding to the filter kernel.
 18. The method ofclaim 1, further comprising the step of selecting a desired frequencyresponse of the final image by selecting the reduced set filter kernel.19. The method of claim 1, further comprising the step of modifying oneof the wavefront coding, converting and processing steps and thenoptimizing and repeating one other of the wavefront coding, convertingand processing steps.
 20. The method of claim 19, the step of processingcomprising utilizing a reduced set filter kernel characterized by aweight matrix.
 21. The method of claim 1, wherein processing the datastream with the reduced set filter kernel comprises processing the datastream with a color-specific filter kernel.
 22. The method of claim 21,the step of converting the optical image to a data stream comprisingoutputting color data from a color digital detector into separatechannels.
 23. The method of claim 1, wherein wavefront coding andprocessing cooperate to preserve image quality over a range of spatialfrequency response in the presence of at least one of: image aberration,depth of focus, depth of field, tolerancing of optics forming theoptical image, tolerancing of mechanics holding the optics, opticalassembly error, noise.
 24. The method of claim 1, wherein processingcomprises utilizing hardware that takes advantage of the reduced setfilter to reduce at least one of multipliers and partial productsummations.
 25. An image processing method, comprising the steps of:wavefront coding a wavefront that forms an optical image; converting theoptical image to a data stream; and processing the data stream with areduced set filter kernel to reverse effects of wavefront coding andgenerate a final image, the step of processing comprising the step ofprocessing the image, for each pixel, with filter tap logic consistingof a shifter.
 26. An image processing method, comprising: wavefrontcoding a wavefront that forms an optical image; converting the opticalimage to a data stream; and utilizing a reduced set gradient filterkernel comprising a series of single-shift differential taps to processthe data stream and reverse effects of wavefront coding, to generate afinal image.
 27. The method of claim 26, further comprising processing asingle summation for any coefficient value of the filter kernel.
 28. Themethod of claim 26, the step of wavefront coding comprising the step ofwavefront coding the wavefront such that a PSF substantially containsinformation of the optical image within a spatial pattern correspondingto the filter kernel.
 29. An image processing method, comprising:wavefront coding a wavefront that forms an optical image; converting theoptical image to a data stream; and processing the data stream with areduced set filter kernel to reverse effects of wavefront coding andgenerate a final image, the step of processing comprising utilizing areduced set distributive filter kernel and a series of distributiveproperty add-scale taps.
 30. The method of claim 29, the step ofprocessing comprising the step of processing a single summation for anycoefficient value of the filter kernel.
 31. The method of claim 29, thestep of wavefront coding comprising the step of wavefront coding thewavefront such that a Point Spread Function (PSF) substantially containsinformation of the optical image within a spatial pattern correspondingto the filter kernel.
 32. An optical imaging system, comprising: awavefront coding element that codes a wavefront forming an opticalimage; a detector for converting the optical image to a data stream; andan image processor for processing the data stream with a reduced setfilter kernel to reverse effects of wavefront coding and generate afinal image, the reduced set filter kernel having coefficientsconsisting of power of two coefficients.
 33. The optical imaging systemof claim 32, the filter kernel being spatially complementary to a pointspread function (PSF) characterizing the optical imaging system informing the optical image.
 34. The optical imaging system of claim 32, aspatial frequency domain version of the filter kernel beingcomplementary to a spatial frequency spectrum of the optical image. 35.The optical imaging system of claim 34, the wavefront coding element,detector and image processor cooperating such that the optical image isspatially correlated to mathematical processing of the data stream withthe reduced set filter kernel.
 36. The optical imaging system of claim35, wherein a rotation of the filter kernel corresponds to like rotationof the wavefront coding element such that the optical image is spatiallycorrelated to mathematical processing of the data stream with thereduced set filter kernel.
 37. The optical imaging system of claim 32,the wavefront coding element comprising an aspheric wavefront codedoptical element.
 38. The optical imaging system of claim 32, thewavefront coded element comprising constant profile path optics.
 39. Theoptical imaging system of claim 38, the reduced set filter kernel beingcomplimentary to a PSF resulting from phase modification of thewavefront through the constant profile path optics.
 40. The opticalimaging system of claim 32, the image processor implementing a filtertap, for each pixel, with logic consisting of a shifter and an adder.41. The optical imaging system of claim 32, the reduced set filterkernel consisting of a plurality of regions wherein coefficients in atleast one of the regions have values of zero.
 42. The optical imagingsystem of claim 41, wherein the image processor comprises hardware thattakes advantage of the reduced set filter coefficients to reduce atleast one of multipliers and partial product summations.
 43. The opticalimaging system of claim 41, the wavefront coding element modifying thewavefront such that a Point Spread Function (PSF) spatially correlatesto the regions of the reduced set filter kernel.
 44. The optical imagingsystem of claim 32, wherein the detector outputs data in the data streamin a non-rectilinear format.
 45. The optical imaging system of claim 44,the format comprising one of a diagonal readout and a circular readout.46. The optical imaging system of claim 44, the reduced set filterkernel being matched to the non-rectilinear format.
 47. The opticalimaging system of claim 46, the image processor remapping data to arectilinear format.
 48. The optical imaging system of claim 44,information within a PSF substantially corresponding to thenon-rectilinear format.
 49. The optical imaging system of claim 32, thereduced set filter kernel comprising a reduced set scaling filter kerneland a series of scaling taps.
 50. The optical imaging system of claim49, the image processor executing a single summation for any coefficientvalue of the filter kernel.
 51. The optical imaging system of claim 32,the reduced set filter kernel being characterized by a weight matrix.52. The optical imaging system of claim 32, wherein the detectorcomprises a color detector and the reduced set filter kernel comprises acolor-specific filter kernel.
 53. The optical imaging system of claim52, the image processor comprising means for separating color data fromthe color digital detector into separate channels.
 54. The opticalimaging system of claim 32, wherein the image processor compriseshardware that takes advantage of the reduced set filter coefficients toreduce at least one of multipliers and partial product summations. 55.An optical imaging system, comprising: a wavefront coding element thatcodes a wavefront forming an optical image; a detector for convertingthe optical image to a data stream; and an image processor forprocessing the data stream with a reduced set filter kernel to reverseeffects of wavefront coding and generate a final image, the imageprocessor implementing a filter tap, for each pixel, with logicconsisting of a shifter.
 56. An optical imaging system, comprising: awavefront coding element that codes a wavefront forming an opticalimage; a detector for converting the optical image to a data stream; andan image processor for processing the data stream with a reduced setfilter kernel to reverse effects of wavefront coding and generate afinal image, the reduced set filter kernel comprising a reduced setgradient filter kernel comprising a series of single-shift differentialtaps.
 57. The optical imaging system of claim 56, the image processorexecuting a single summation for any coefficient value of the filterkernel.
 58. The optical imaging system of claim 56, the wavefront codingelement modifying the wavefront such that a PSF contains substantiallyall information within a spatial pattern corresponding to the filterkernel.
 59. An optical imaging system, comprising: a wavefront codingelement that codes a wavefront forming an optical image; a detector forconverting the optical image to a data stream; and an image processorfor processing the data stream with a reduced set filter kernel toreverse effects of wavefront coding and generate a final image, thereduced set filter kernel comprising a reduced set distributive filterkernel having coefficients consisting of power of two coefficients. 60.The optical imaging system of claim 59, the image processor executing asingle summation for any coefficient value of the filter kernel.
 61. Theoptical imaging system of claim 59, the reduced set filter kernelcomprising a series of distributive property scale-add taps.
 62. Anelectronic device, comprising a camera having: a wavefront codingelement that phase modifies a wavefront that forms an optical imagewithin the camera; a detector for converting the optical image to a datastream; and an image processor for processing the data stream with areduced set filter kernel to reverse effects of wavefront coding andgenerate a final image, the reduced set filter kernel havingcoefficients consisting of power of two coefficients.
 63. The electronicdevice of claim 62, wherein the electronic device forms one of a cellphone and teleconferencing apparatus.
 64. A system for generating anelectronic image, comprising: constant profile path optics that code awavefront forming an optical image; a detector for converting theoptical image to a data stream; and an image processor for processingthe data stream with a reduced set filter kernel having coefficientsconsisting of power of two coefficients in order to reverse effects ofwavefront coding and generate the electronic image.
 65. An imageprocessing method, comprising the steps of: wavefront coding a wavefrontthat forms an optical image; converting the optical image to a datastream; and processing the data stream with a reduced set filter kernelto reverse effects of wavefront coding and generate a final image, thestep of wavefront coding comprising utilizing an aspheric opticalelement to implement the wavefront coding, the aspheric optical elementhaving one of the group consisting of fourfold symmetry and fivefoldsymmetry.
 66. The method of claim 65, the step of processing comprisingrotating the reduced set filter kernel in accordance with rotation ofthe aspheric optical element.
 67. An optical imaging system, comprising:a wavefront coding element that codes a wavefront forming an opticalimage; a detector for converting the optical image to a data stream; andan image processor for processing the data stream with a reduced setfilter kernel to reverse effects of wavefront coding and generate afinal image, the wavefront coding element having one of fourfoldsymmetry and fivefold symmetry.